A Stochastic Articulatory-to-acoustic Mapping as a Basis for Speech Recognition

Hidden Markov models (HMMs) of speech acoustics are the current state-of-the-art in speech recognition, but these models bear little resemblance to the processes underlying speech production (Lee, 1989). In this respect, using an HMM to model speech acoustics is like using a Gaussian distribution to model data generated by a Poisson process – to the extent that the model is not an accurate representation of generating process, the accuracy of the model, and the meaning of the inferred parameters, is limited. Of This model mismatch likely contributes to the fact that state-ofthe-art recognition performance (word accuracy) on recorded telephone conversations is only around 60-65%. There have been recent attempts to create stochastic models of speech acoustics make more realistic assumptions about the mechanisms underlying speech production (Bakis, 1991; Deng, 1998; Hogden, 1998; Picone et al., 1999). In this paper we describe two stochastic models of speech “production Conditional Observable Maximum Likelihood Mapping (CO-MALCOM) and its predecessor, Maximum Likelihood Continuity Mapping (MALCOM). The main component of both of these models is a stochastic mapping between speech acoustics and speech articulation. A counter-intuitive aspect of the stochastic mapping is that the parameters of the mapping can be found using only acoustic data. While most speech researchers are familiar with the fact that HMM parameters can be estimated from acoustics alone, many still find it surprising that the mapping between speech acoustics and speech articulator positions (positions of the tongue, jaw, lips . ..) can be found without articulator position measurements. Nonetheless, there are theoretical and experimental reasons to believe that MALCOM and it’s allies learn a stochastic mapping between articulator positions and speech acoustics. Furthermore, CO-MALCOM can be combined with standard speech recognition algorithms to form a speech recognition approach based on a production model. Results of experiments related to MALCOM are summarized, and the COMALCOM extension is described. BACKGROUND: STATE-OF-THE-ART SPEECH RECOGNITION In many realistic domains, automatic speech recognition performance is inadequate. To be concrete, at the National Institute of Standards and Technology 1998 HUB-5 Speech Recognition Evaluation, state-of-the-art systems had about a 6070-65% word recognition rate on “------’ casual